Jan Beulich [Thu, 1 Dec 2011 16:57:54 +0000 (17:57 +0100)]
x86/microcode: enable boot time (pre-Dom0) loading
Largely as a result of the continuing resistance of Linux maintainers
to accept a microcode loading patch for pv-ops Xen kernels, this
follows the suggested route and provides a means to load microcode
updates without the assistance of Dom0, thus also addressing eventual
problems in the hardware much earlier.
This leverages the fact that via the multiboot protocol another blob
of data can be easily added in the form of just an extra module. Since
microcode data cannot reliably be recognized by looking at the
provided data, this requires (in the non-EFI case) the use of a
command line parameter ("ucode=<number>") to identify which of the
modules is to be parsed for an eventual microcode update (in the EFI
case the module is being identified in the config file, and hence the
command line argument, if given, will be ignored).
This required to adjust the XSM module determination logic accordingly.
The format of the data to be provided is the raw binary blob already
used for AMD CPUs, and the output of the intel-microcode2ucode utility
for the Intel case (either the per-(family,model,stepping) file or -
to make things easier for distro-s integration-wise - simply the
concatenation of all of them).
In order to not convert the spin_lock() in microcode_update_cpu() (and
then obviously also all other uses on microcode_mutex) to
spin_lock_irqsave() (which would be undesirable for the hypercall
context in which the function also runs), the boot time handling gets
done using a tasklet (instead of using on_selected_cpus()).
Signed-off-by: Jan Beulich <jbeulich@suse.com> Acked-by: Keir Fraser <keir@xen.org>
Jan Beulich [Thu, 1 Dec 2011 16:55:26 +0000 (17:55 +0100)]
x86: consolidate microcode loading code
- memory was leaked on a CPU offline/online cycle (including S3)
- memory was leaked on AMD systems when microcode_update() ran a 2nd
time with the same data that was used on the first run
- microcode never got restored on APs during S3 resume (or post-boot
onlining of a CPU that was also online when microcode_update() first
ran [in the event the prior microcode update got lost intermediately,
which supposedly shouldn't happen]); this will still be the case when
no other online CPU has an identical signature (which however is now
consistent with bringing up such a CPU the very first time)
- resume was unimplemented in the AMD case
- there was a race between microcode_update_cpu() and
microcode_resume_cpu()
This also moves vendor specific type declarations to the vendor source
file and sets the stage for boot time microcode loading (i.e. without
Dom0 involvement).
Signed-off-by: Jan Beulich <jbeulich@suse.com> Acked-by: Keir Fraser <keir@xen.org>
Ian Jackson [Thu, 1 Dec 2011 16:18:58 +0000 (16:18 +0000)]
tools: Switch to system libaio (again)
The test system problems which prompted 24233:a9c67c2daf4b (itself a
tiny partial revert of 24184:4ecd3615e726 / 24186:7aa5838499d1) have
now been resolved, we think. So revert 24233.
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
Instead of sending dirty pages of guest memory as-is, use a simple compression
algorithm that sends a RLE-encoded XOR of the page against its last sent copy.
A small LRU cache is used to hold recently dirtied pages. Pagetable pages are
sent as-is, as they are canonicalized at sender side and uncanonicalized at
receiver.
[ Fixed up a conflict in sg_save_restore.h. I had to increase the
ID values used from -11 and -12 to -12 and -13 because -11 had
been taken by ..._HVM_VIRIDIAN in the meantime. -iwj ]
Signed-off-by: Shriram Rajagopalan <rshriram@cs.ubc.ca> Acked-by: Brendan Cully <brendan@cs.ubc.ca> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com> Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
tools/libxc: introduce xc_memalign in xc_{minios,linux,solaris,netbsd}.c
Move (page aligned) buffer allocations in {os}_privcmd_alloc_hypercall_buffer
into a global function xc_memalign. This API is also used by Remus
compression code to allocate compression caches that need to be page aligned.
Signed-off-by: Shriram Rajagopalan <rshriram@cs.ubc.ca> Acked-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Brendan Cully <brendan@cs.ubc.ca> Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
The p2m audit code doesn't even compile, let alone work. It also
partially supports ept. Make it:
- compile
- lay groundwork for eventual ept support
- move out of the way of all calls and turn it into a domctl. It's
obviously not being used by anybody presently.
- enable it via said domctl
Signed-off-by: Andres Lagar-Cavilla <andres@lagarcavilla.org> Signed-off-by: Tim Deegan <tim@xen.org> Acked-by: Tim Deegan <tim@xen.org> Committed-by: Tim Deegan <tim@xen.org>
Tim Deegan [Thu, 1 Dec 2011 14:17:16 +0000 (14:17 +0000)]
x86/mm: Don't lose track of the log dirty bitmap
hap_log_dirty_init unconditionally sets the top of the log dirty
bitmap to INVALID_MFN. If there had been a bitmap allocated, it is
then leaked, and the host crashes on an ASSERT when the domain is
cleaned up.
Signed-off-by: Tim Deegan <tim@xen.org> Acked-by: Andres Lagar-Cavilla <andres@lagarcavilla.org> Committed-by: Tim Deegan <tim@xen.org>
x86/mm: Don't trigger unnecessary shadow scans on p2m entry update
When updating a p2m entry, the hypervisor scans all shadow pte's to find
mappings of that gfn and tear them down. This is avoided if the page count
reveals that there are no additional mappings. The current test ignores the
PGC_allocated flag and its effect on the page count.
Signed-off-by: Andres Lagar-Cavilla <andres@lagarcavilla.org> Signed-off-by: Adin Scannell <adin@scannell.ca> Signed-off-by: Tim Deegan <tim@xen.org> Committed-by: Tim Deegan <tim@xen.org>
Liu, Jinsong [Thu, 1 Dec 2011 11:22:43 +0000 (12:22 +0100)]
X86: Disable PCID/INVPCID for dom0
PCID (Process-context identifier) is a facility by which a logical
processor may cache information for multiple linear-address spaces.
INVPCID is an new instruction to invalidate TLB. Refer latest Intel SDM
http://www.intel.com/content/www/us/en/processors/architectures-software-developer-manuals.html
We disable PCID/INVPCID for dom0 and pv. Exposing them into dom0 and pv
may result in performance regression, and it would trigger GP or UD
depending on whether platform suppport INVPCID or not.
This patch disables PCID/INVPCID for dom0.
Signed-off-by: Liu, Jinsong <jinsong.liu@intel.com> Committed-by: Jan Beulich <jbeulich@suse.com>
Jan Beulich [Thu, 1 Dec 2011 07:51:35 +0000 (08:51 +0100)]
x86/emulator: cleanup
Utilize some of the additions in the prior patches to clean up other
code:
- keep track of REP prefixes in only one variable
- use REX_W in a few more places (instead of a literal number)
Signed-off-by: Jan Beulich <jbeulich@suse.com> Acked-by: Keir Fraser <keir@xen.org>
Jan Beulich [Thu, 1 Dec 2011 07:50:49 +0000 (08:50 +0100)]
x86/emulator: properly handle lzcnt and tzcnt
These instructions are prefix selected flavors of bsf and bsr
respectively, and hence the presences of the F3 prefix must be handled
in the emulation code in order to avoid running into problems on newer
CPUs.
Jan Beulich [Thu, 1 Dec 2011 07:48:14 +0000 (08:48 +0100)]
x86/emulator: generalize movq emulation (SSE2 and AVX variants)
Extend the existing movq emulation to also support its SSE2 and AVX
variants, the latter implying the addition of VEX decoding. Fold the
read and write cases (as most of the logic is identical), and add
movntq and variants (as they're very similar).
Extend the testing code to also exercise these instructions.
Signed-off-by: Jan Beulich <jbeulich@suse.com> Acked-by: Keir Fraser <keir@xen.org>
Keir Fraser [Thu, 1 Dec 2011 00:59:58 +0000 (16:59 -0800)]
xenpaging: Fix c/s 23507:0a29c8c3ddf7 ("update machine_to_phys_mapping[] during page deallocation")
This patch clobbers page owner in free_heap_pages() before we are
finished using it. This means that a subsequent test to determine
whether it is safe to avoid safety TLB flushes incorrectly always
determines that it is safe to do so.
The fix is simple: we can defer the original patch's work until after
we are done with the page-owner field.
Thanks to Christian Limpach for spotting this one.
Olaf Hering [Wed, 30 Nov 2011 15:08:53 +0000 (07:08 -0800)]
mem_event: move mem_event_domain out of struct domain
An upcoming change may increase the size of mem_event_domain. The result
is a build failure because struct domain gets larger than a page.
Allocate the room for the three mem_event_domain members at runtime.
v2:
- remove mem_ prefix from members of new struct
Signed-off-by: Olaf Hering <olaf@aepfle.de> Committed-by: Keir Fraser <keir@xen.org>
Paul Durrant [Wed, 30 Nov 2011 14:53:36 +0000 (06:53 -0800)]
hvmloader: Allocate an 8 byte buffer to contain the VM generation id
and populate it at boot time with a value read from
"platform/generation_id". Also add code to libxl to populate this
xenstore key with the value of a new 'generation_id' parameter in the
VM config file. Populate the ADDR package of VM_Gen_Counter ACPI
device such that the first integer evaluates to the low order 32 bits
of the buffer address and the second integer evaluates to the high
order 32 bits of the buffer address.
Signed-off-by: Paul Durrant <paul.durrant@citrix.com> Committed-by: Keir Fraser <keir@xen.org>
Paul Durrant [Wed, 30 Nov 2011 14:42:04 +0000 (06:42 -0800)]
x86/hvm: Re-instate HVM IRQ debug code and add keyhandler.
I found this patch useful a couple of times while trying to debug the
viridian problem. irq_dump() was #ifdef-ed out so this patch puts it
back and registers a handler on the 'I' key to iterate over all HVM
domains and call it.
Signed-off-by: Paul Durrant <paul.durrant@citrix.com> Committed-by: Keir Fraser <keir@xen.org>
Ian Campbell [Tue, 29 Nov 2011 16:23:27 +0000 (16:23 +0000)]
docs: improve index.html generation
Include hypercall documentation, fixing link generation for top level links to
use the INDEX.
Allow subsection links to be renamedi in the INDEX too.
Strip .txt suffixes as well as .html ones by moving the regex to the right
place instead of placing the literal text "(?:html|txt)" into the backlink to
the top level page. (oops)
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Tue, 29 Nov 2011 15:48:08 +0000 (15:48 +0000)]
docs/html/: Arrange for automatic build of hypercall docs
- Use index.html rather than a stamp file.
- Automatically generate dependencies.
- Wire into the docs build system
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com> Committed-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Tue, 29 Nov 2011 15:48:07 +0000 (15:48 +0000)]
docs/html/: Generate an "index.html" for hypercall docs
Generate an "index.html" containing various useful links.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com> Committed-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Tue, 29 Nov 2011 15:48:07 +0000 (15:48 +0000)]
docs/html/: Annotations for two hypercalls
Add annotations for a couple of the hypercalls:
HYPERVISOR_set_trap_table
HYPERVISOR_mmu_update
We do this by
* annotating the list of #defines for hypercall numbers
* annotating the list of error values
* providing a function prototype for the systematically-named functions
The header generator does the rest.
This exercise revealed a couple of infelicities:
* In the actual source code, do_mmu_update is defined to return an int
and do_set_trap_table a long. However both functions return either
-Efoo (on error) or 0 for success.
* The error numbers are defined only in the private header file
xen/include/xen/errno.h and then only with names which will
typically clash with other projects. It would be nice to include a
public version of this header which defines XEN_E*. But for now
we run xen-headers on errno.h too.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com> Committed-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson [Tue, 29 Nov 2011 15:48:06 +0000 (15:48 +0000)]
docs/html/: Initial cut of header documentation massager
"xen-headers" generates HTML from header files. So far this generates
just some type cross-references, if you say
make -C docs html/hypercall/stamp
An index page, proper wiring into the build system, and a few more
annotations in the headers, and will be forthcoming.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com> Committed-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Campbell [Tue, 29 Nov 2011 14:19:59 +0000 (14:19 +0000)]
docs: improve documantion of xl's viridian option
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Committed-by: Ian Jackson <ian.jackson.citrix.com> Acked-by: Ian Jackson <ian.jackson.citrix.com>
Ian Campbell [Tue, 29 Nov 2011 14:17:27 +0000 (14:17 +0000)]
docs: xlexample.hvm: mention the viridian setting.
Turning this on for Windows guests is recommended.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Committed-by: Ian Jackson <ian.jackson.citrix.com> Acked-by: Ian Jackson <ian.jackson.citrix.com>
Ian Campbell [Tue, 29 Nov 2011 14:17:27 +0000 (14:17 +0000)]
docs: install txt files as html
A browser will display them just fine.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Committed-by: Ian Jackson <ian.jackson.citrix.com> Acked-by: Ian Jackson <ian.jackson.citrix.com>
Ian Campbell [Tue, 29 Nov 2011 14:17:27 +0000 (14:17 +0000)]
libxlu: add xlu_cfg_get_list_as_string_list
Returns a cfg list as a libxl_string_list.
Use this to simplify the parsing of device model extra args.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Committed-by: Ian Jackson <ian.jackson.citrix.com> Acked-by: Ian Jackson <ian.jackson.citrix.com>
Ian Campbell [Tue, 29 Nov 2011 14:17:27 +0000 (14:17 +0000)]
docs: move user and interface .tex documents under reference.
Taking over the top level "user" entry with a relatively obsolete document is a
bit of an annoyance but these docs are not so out of date that they should be
deleted. Move them out of the top-level instead.
(the original motivation here was to allow for user/xl-domain-cfg.markdown but
we have since decided to go with man/xl.cfg.pod.5 instead so perhaps this is a
waste of time)
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Committed-by: Ian Jackson <ian.jackson.citrix.com> Acked-by: Ian Jackson <ian.jackson.citrix.com>
Ian Campbell [Tue, 29 Nov 2011 14:17:27 +0000 (14:17 +0000)]
docs: generate an index for the html output
nb: I'm not a Perl wizard...
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Committed-by: Ian Jackson <ian.jackson.citrix.com> Acked-by: Ian Jackson <ian.jackson.citrix.com>
Ian Campbell [Tue, 29 Nov 2011 14:17:27 +0000 (14:17 +0000)]
docs: tweak markup and wording of qemu upstream doc slightly
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Committed-by: Ian Jackson <ian.jackson.citrix.com> Acked-by: Ian Jackson <ian.jackson.citrix.com>
Ian Campbell [Tue, 29 Nov 2011 15:18:18 +0000 (15:18 +0000)]
docs: remove non-breaking spaces from sedf_scheduler_mini-HOWTO.txt
This document contains several 0xa0 characters (non-breaking spaces). These do
not display correctly in (some) terminals or when the document is rendered by (some)
browsers. Re-encode them as spaces.
I'm not confident that this change will make it through being encoded as a patch
and sent through email. Its effect can be replicated with:
Ian Campbell [Tue, 29 Nov 2011 14:17:27 +0000 (14:17 +0000)]
xlu: add "dont_warn" to xlu_cfg_*
I want it for get_long but we might as well have it everywhere.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Committed-by: Ian Jackson <ian.jackson.citrix.com> Acked-by: Ian Jackson <ian.jackson.citrix.com>
Ian Campbell [Tue, 29 Nov 2011 14:17:27 +0000 (14:17 +0000)]
xl: the name field in a guest config file is mandatory
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Committed-by: Ian Jackson <ian.jackson.citrix.com> Acked-by: Ian Jackson <ian.jackson.citrix.com>
Ian Campbell [Tue, 29 Nov 2011 14:17:27 +0000 (14:17 +0000)]
docs: add a document describing the xl cfg file syntax
Based on an initial version by Ian Jackson.
I believe that all keys are now present in the document although there are are
various omissions in the actual documentation of them. Hopefully however this
covers the majority of the most interesting keys.
Spice section: Signed-off-by: Zhou Peng <zhoupeng@nfs.iscas.ac.cn> Committed-by: Ian Jackson <ian.jackson.citrix.com> Acked-by: Ian Jackson <ian.jackson.citrix.com>
The rest: Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Committed-by: Ian Jackson <ian.jackson.citrix.com> Acked-by: Ian Jackson <ian.jackson.citrix.com>
Ian Campbell [Tue, 29 Nov 2011 14:17:27 +0000 (14:17 +0000)]
docs: install html and txt versions of manpages
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Committed-by: Ian Jackson <ian.jackson.citrix.com> Acked-by: Ian Jackson <ian.jackson.citrix.com>
Ian Campbell [Tue, 29 Nov 2011 14:17:27 +0000 (14:17 +0000)]
README: add markdown to dependency list
although this tool is strictly speaking optional we are providing various user
docs in this format so increase the changes that they will install it.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Committed-by: Ian Jackson <ian.jackson.citrix.com> Acked-by: Ian Jackson <ian.jackson.citrix.com>
Ian Campbell [Tue, 29 Nov 2011 14:17:27 +0000 (14:17 +0000)]
docs: xlexample.hvm is missing "builder = 'hvm'"
This is rather critical...
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Committed-by: Ian Jackson <ian.jackson.citrix.com> Acked-by: Ian Jackson <ian.jackson.citrix.com>
Note that "PythonInXlConfig" never existed in the old wiki and does not exist
in the new. This reference was introduced by 22735:cb94dbe20f97 and was
supposed to have been written prior to the 4.1 release. I have transitioned it
anyway but it's not clear how valuable the message actually is. Perhaps we
should just remove that aspect of it?
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Committed-by: Ian Jackson <ian.jackson.citrix.com> Acked-by: Ian Jackson <ian.jackson.citrix.com>
Ian Campbell [Mon, 28 Nov 2011 17:42:40 +0000 (17:42 +0000)]
libxl: propagate error from tap_ctl_spawn.
Failure here means that a disk will not be correctly setup. I briefly
scanned tools/blktap2/control.c for other goto constructs which did not
set their err variable but didn't see any others.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Mon, 28 Nov 2011 17:19:28 +0000 (17:19 +0000)]
libxl: open logs with O_APPEND
Logfiles should always be opened for append.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com> Acked-by: Ian Campbell <ian.campbell@citrix.com> Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian Jackson [Mon, 28 Nov 2011 11:57:23 +0000 (11:57 +0000)]
tools: Revert to our built-in aio
These two changesets:
24184:4ecd3615e726 tools: use system installed libaio by default.
24186:7aa5838499d1 tools: use system libaio for blktap1 as well.
cause HVM guest installs (both Windows and Redhat) to fail on Debian
squeeze with xl. So change the default for now.
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
Paul Durrant [Fri, 25 Nov 2011 15:30:41 +0000 (15:30 +0000)]
Fix save/restore for HVM domains with viridian=1
xc_domain_save/restore currently pay no attention to
HVM_PARAM_VIRIDIAN which results in an HVM domain running a recent
version on Windows (post-Vista) locking up on a domain restore due to
EOIs (done via a viridian MSR write) being silently dropped. This
patch adds an extra save entry for the viridian parameter and also
adds code in the viridian kernel module to catch attempted use of
viridian functionality when the HVM parameter has not been set.
Signed-off-by: Paul Durrant <paul.durrant@citrix.com> Committed-by: Keir Fraser <keir@xen.org>
Olaf Hering [Sun, 20 Nov 2011 16:02:50 +0000 (17:02 +0100)]
xenpaging: improve policy mru list handling
Without this change it is not possible to page-out all guest pages, then
trigger a page-in for all pages, and then page-out everything once
again. All pages in the mru list can not be paged out because they
remain active in the internal bitmap of paged pages.
Use the mru list only if the number of paged-out pages is larger than
the mru list. If the number is smaller, start to clear the mru list. In
case the number of paged-out pages drops to zero the mru list and the
internal bitmap will be empty as well.
Also add a new interface for dropped pages. If a gfn was dropped there
is no need to adjust the mru list because dropping a page is not usage
of a page.
Signed-off-by: Olaf Hering <olaf@aepfle.de> Committed-by: Ian Jackson <ian.jackson.citrix.com>
Olaf Hering [Sun, 20 Nov 2011 16:02:48 +0000 (17:02 +0100)]
xenpaging: add cmdline interface for pager
Introduce a cmdline handling for the pager. This simplifies libxl support,
debug and mru_size are not passed via the environment anymore.
The new interface looks like this:
xenpaging [options] -f <pagefile> -d <domain_id>
options:
-d <domid> --domain=<domid> numerical domain_id of guest. This option is required.
-f <file> --pagefile=<file> pagefile to use. This option is required.
-m <max_memkb> --max_memkb=<max_memkb> maximum amount of memory to handle.
-r <num> --mru_size=<num> number of paged-in pages to keep in memory.
-d --debug enable debug output.
-h --help this output.
Signed-off-by: Olaf Hering <olaf@aepfle.de> Committed-by: Ian Jackson <ian.jackson.citrix.com>
Olaf Hering [Sun, 20 Nov 2011 16:02:45 +0000 (17:02 +0100)]
xenpaging: use guests tot_pages as working target
This change reverses the task of xenpaging. Before this change a fixed number
of pages was paged out. With this change the guest will not have access to
more than the given number of pages at the same time.
Signed-off-by: Olaf Hering <olaf@aepfle.de> Committed-by: Ian Jackson <ian.jackson.citrix.com>
Olaf Hering [Sun, 20 Nov 2011 16:02:39 +0000 (17:02 +0100)]
xenpaging: move page add/resume loops into its own function.
Move page resume loop into its own function.
Move page eviction loop into its own function.
Allocate all possible slots in a paging file to allow growing and
shrinking of the number of paged-out pages. Adjust other places to
iterate over all slots.
This change is required by subsequent patches.
v2:
- check if victims allocation succeeded
Signed-off-by: Olaf Hering <olaf@aepfle.de> Committed-by: Ian Jackson <ian.jackson.citrix.com>
projects / xen.git / log